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ABSTRACT 

The study examined how response rate to a mailed 
questionnaire was influenced by the socioeconomic status (SES) of 
respondents (high, middle, or low) ; amount of payment of a monetary 
incentive (none, $1, or $2); and time of payment (prepaid or promised 
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in Cache County, Utah at each SES level. Higher payment elicited more 
returns than lower payment, prepayment elicited more than promised 
payment, and high-SES subjects returned more questionnaires than 
low-SES subjects. Also, there was an interaction between SES and 
payment timing. Prepayment elicited a much higher rate than promised 
payment for low-SES subjects, while pre- versus promised payment made 
little difference for high-SES subjects. (Contains 9 references and 3 
tables.) (Author/SLD) 
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Abstract 

The study examined how response rate to a mailed questionnaire was influenced by 
socioeconomic status (SES) of respondents (high, middle, or low); amount of payment of a 
monetary incentive (none, $1, or $2); and time of payment (prepaid or promised upon return 
of completed questionnaire). Higher payment elicited more returns than lower payment, 
prepayment elicited more than promised payment, and high-SES subjects returned more 
questionnaires than low-SES subjects. Also, there was an interaction between SES and 
payment timing: Prepayment elicited a much higher return rate than promised payment for 
low-SES subjects, while pre- vs. promised payment made little difference for high-SES 
subjects. 
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Effects of Respondents' Socioeconomic Status and Timing 
and Amount of Incentive Payment on Mailed Questionns»^Te Response Rates 
The mailed questionnaire is one of the most economical methods of data collecting, 
especially when the target universe for the survey is spread over a large geographical area. 
The validity of conclusions reached tiirough the use of tiiis instrument is, however, often 
questionable because of iow response rates and the resultant possibility of bias. In trying to 
overcome this problem, much research has been done on variables thought to influence 
response rates. 

One of the most widely studied of these variables has been the use of monetary 
incentives, with the general consensus being that their inclusion does succeed in improving 
response rates (Armstrong, 1975). In variations of this theme, researchers have also looked 
at differences in response rates related to the amount of the incentive (Doob, Freedman, & 
Carlsmith, 1973; Schewe & Coumoyer, 1976), and the time of incentive payment 
(prepayment vs. payment promised on return of a completed survey) (Skinner, Ferrell, & 
Pride, 1984; W'^iss, Friedman, & Shoemaker, 1985). These findings tentatively suggest that 
prepayment of the incentive results in a higher return rate than a promised payment, and that 
. response rates rise with the amount of the incentive. Unfortunately, most previous work has 
been marked by such methodological flaws as convenience sampling, confounding of 
independent variables, and/or failure to conduct a nonresponse bias check, which render 
findings inconclusive. 

Moreover, little research has been done on the socioeconomic status (SES) of the 
respondent, which has been viewed as a potentially important variable related to the use of 
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monetary incentives to increase response rates. Given the move away from mass marketing 
in favor of targeting specific niches, many data gathering efforts' are aimed only at specific 
social strata. It is important, therefore, to understand the influence of SES on response rates, 
both alone and in conjunction with payment variables. 

A review of research by the first author yielded only one prior study of the joint 
impact of SES and monetary incentives on response rate. Gelb (1975) distributed 
questionnaires to grocery store shoppers in two neighborhoods, one middle class and one 
lower class. Half the sample in each store received a questionnaire with 50 cents attached, 
while half were promised 50 cents upon return of the questionnaire. The rate of return from 
those offered immediate payment was 54% among middle-class shoppers and 15% among 
lower-class shoppers. For those promised payment, the return rate was 45% among middle- 
class shoppers and 25% among lower class shoppers. 

While Gelb (1975) is an important study, it does have methodological weaknesses, 
specifically the use of convenience sampling rather than random sampling. The present study 
corrects this flaw while measuring the individual and joint effect on response rate of SES, 
time of incentive payment, and amount of payment. The relative cost effectiveness of 
various payment methods is also considered. 

Method 

Population and Sampling 

The target population was adult residents of Cache County, Utah. Cache County is in 
a valley 80 miles north of Salt Lake City and consists of the city of Logan (population 
35,000) and surrounding communities and rural areas. Logan is the site of Utah State 



University, which is the primary employer in the valley; in addition, dairy farming, 
ranching, and tourism are other important employment bases in the area. 

Because the purpose of the study was to determine the relationship between SES and 
response rates, the first procedure was to classify subjects into high-, medium-, and low-SES 
categories. While this could have been dore using completed questionnaires, past experience 
has shown that many respondents return questionnaires with all but the demographic 
questions answered, making after-the-fact classification impossible. Thus, in order to 
classify respondents before mailing the questionnaires, SES information was obtained using 
the following sampling procedure. 

Phone survey. A random sample of 700 telephone numbers was selected from the 
Cache County telephone directory. (While directory sampling is often faulted for its 
omission of unlisted numbers, this problem affected less than 6% of the county population 
according to the local telephone company.) The survey had a dual purpose. One was to 
collect data on citizens' attitudes toward wildlife management. The second purpose, which 
was pertinent to the present study, was to elicit SES-related information. It had been decided 
to mail the main survey to 125 subjects in each SES level, for a total of 375; 580 
respondents were contacted in the phone survey to ensure; reaching this number, based on 
estimates of noncooperation experienced in prior, similar surveys in this area. The telephone 
survey was carried out by three undergraduate students trained specifically for this task. 
During these telephone calls, they obtained the SES data discussed hereafter. 

SES Classification. SES indices are usually formed by combining all or any two of 
the variables education, occupation, aid household income. For the present study, education 



and household income were used. Scores were assigned to different education and income 
levels based on their distributions as contained in the Statistical Abstract of the United States 
(U.S. Department of Commerce, 1989) and on procedures used by the Census Bureau (U.S. 
Bureau of the Census, 1963). Once a respondent's scores for education and income were 
determined, they were summed to form a composite SES score. A frequency distribution 
was then plotted for the entire sample and three levels of SES were defined according to 
naturally occurring clusters. The above procedure resulted in 125 subjects in the high- and 
low-SES groups, but more than 125 in the middle-SES group. The final selection of 125 in 
the middle-SES group was then done randomly. 

As an external validation of the classification procedure, results were compared with 
federal poverty standards (U.S. Department of Commerce, 1989). Approximately 92% of 
subjects in the low-SES group met these standards. 
Research Design 

The 125 subjects in each SES group (low, middle, and high) were randomly assigned 
to one of five incentive conditions: prepaid $1, prepaid $2, promised $1, promised $2, or 
an unpaid control group. 
Mail Survey Procedure 

A reciprocal arrangement was made with Utah State University's Extension Service, 
which was interested in conducting a survey of family and economic well-being in Cache 
County, such that all survey costs were borne by the Extension Service, in exchange for the 
first author carrying out all survey activities and turning the completed questionnaires over to 
the Service. 
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Once the seven-page questionnaire had been developed, a mailing packet was 
assembled for each subject consisting of the questionnaire, a cover letter, and a business- 
reply envelope in which to return the questionnaire. The cover letter briefly explained the 
information requested in the questionnaire and contained a promise of anonymity. The letter 
also explained either the promise or presence of the monetary incentive, whether payment 
was being made now or was promised for later, and the amount of the incentive. For the 
appropriate groups, the packet also included the amount of monetary incentive that 
constituted the experimental tireatment. The cover letter was printed on Extension Service 
stationery and was signed by Extension Service officials. 

Respondents were given approximately 3 weeks to reply, by which time the return 
rate had dropped to one or two per day. Though Extension Service officials carried out two 
follow-up mailings, the present study used only responses to the first mailing. 

Results 

Table 1 shows the percentage of questionnaires returned for each experimental 
condition at each SES level. 



Insert Table 1 about here 



Data were coded dichotomously: Either the subject returned or did not return the 
questionnaire. These data were then analyzed in a 3 (SES levels) x 2 (amount of payment) x 
2 (time of payment) fixed factors ANOVA, using the SPSS computer package. (The contirol 
group was not included in this analysis, because it has empty cells - the "zero" amount of 
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payment group cannot be divided into two time-of-payment levels. The control group was 
used, however, in some post-hoc analyses to be discussed later.) The ANOVA results are 
summarized in Table 2. 



Insert Table 2 about here. 



Tests of the main effects revealed that the return rate differences for amount of 
payment and time of payment were statistically significant at g < .05 and e < -OOl. 
respectively, with the higher amount and the prepayment resulting in higher return rates. 
The return rates for the three SES levels also differed to a statistically significant degree at p 
< .01; a post-hoc Tukey test revealed that the difference in return rates for the high (72%) 
and low (48%) SES groups was the source of the statistical significance. 

As Table 3 shows, when the return rates are collapsed across incentive amounts, (i.e., 
$1 and $2 groups are combined), the time of payment appears to have had Jitfle effect on the 
return rate for high-SES subjects. However, timing was found to have a substantial effect on 
the return rates for middle and low-SES subjects, with prepaid subjects returning a far higher 
percentage of questionnaires than those promised payment. 



Insert Table 3 about here 



Tests of the two-way interactions showed a statistically signifi(^t interaction between 
SES and time of payment, and post-hoc tests of simple main effects revealed that the effect 



of time of payment was not statistically significant for the high-SES group, but was 
statistically significant for the middle-SES and lovz-SES groups (F = 5.02, 2 < .03, and F 
= 14.78, g < .001, respectively). 

The three-way SES by Time by Amount interaction was not statistically significant; 
pooling the three-way sum of squares into the error sum of squares did not affect any of the 
results with respect to statistical significance. 

Discussion 

The present study reaffirms what previous research has shown in that each of the 
three independent variables studied ~ SES, amount of payment, and timing of payment ~ 
was related to response rates to mailed questionnaires. However, the present study has 
extended previous work by using metliodology based on random samples and a design that 
explored all main effects and interactions. 

An important finding was the interaction between SES and time of payment: High- 
SES subjects returned the questionnaires at a similarly high rate whether the payment was 
made in advance or promised, but for middle- and low-SES subjects, prepayment elicited a 
higher rate of return. The difference in return rates based on timing of payment was 
particularly large for low-SES subjects (66% for prepaid vs. 30% for those promised 
payment). 

These findings have obvious implications for survey researchers working with 
populations that include people of lower socioeconomic status: prepaying the incentive can 
boost response rates to levels comparable to those of high- 1 middle-SES populations, 
while promising later payment of $1 or $2 (upon return of a completed survey), is no better 
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-- and perhaps worse ~ than not paying at all. For high-SES groups, it is questionable 
whether one needs to pay an incentive at all; in the present study, high-SES subjects v/ho 
were paid had a 72% return rate, compared with 60% for unpaid controls, a substantial 
difference. However, at the sample size used in the present study, the difference was not 
statistically significant at p < .05. Further research on high-SES populations should clarify 
this issue. In mixed-SES populations, tailoring incentives to respondents' SES level might be 
seen .is a way to maximize cost-effectiveness, although this raises ethical issues as well as 
internal validity problems. 

This study is not without limitations. The final sample was selected only if they 
answered SES-related questions in the prior telephone survey; although most respondents did 
answer these questions, the fact that some chose not to makes this technically a volunteer 
sample. Also, although all that was affordable in the present study, 25 subjects per cell is a 
relatively small sample size. Finally, the mailed questionnake was seven pagos long; it is 
possible that the effects of SES, amount of payment, and timing of payment on response 
rates would be different for a shorter questionnaire. Dillman (1978), however, has. reported 
that questionnaires up to 12 pages long do not adversely affect response rates. 

Future research in this area might focus on replicating the findings using different 
questionnaire lengths and/or topics, different incentive levels, and populations from more 
urban settings and from different geographic regions. 
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Table 1. 



Percent returned by SES. time of payment, and amount of payment 
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SES 



Prepaid Promised 
$1 $2 $1 $2 Control Overall 



High 

Middle 

Low 



68 
68 
60 



80 
68 
72 



64 
40 
20 



76 
52 
40 



60 
52 
32 



70 
56 
45 



Overall 



65 



73 



41 



56 



48 



57 



Note : Percentages based on n = 25 per cell. 
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Table 2. 

ANQVA results for effects of SES. amount of pay m ent, and time of payment on rate of return 



Factor 


SS 


df 


MS 


F 


2 


SES 


3.41 


2 


1.70 


7.78 


.001 


Amount 


■:96 


1 


.96 


'■'■AO 


.037 


Time 


3.63 


1 


3.63 


16.58 


nnn 


SES X Amount 


.13 


2 


.06 


.29 


.749 


SES X Time 


1.62 


2 


.81 


3.70 


.026 


rVllLUUILL A. lilllC 


.08 


1 


.08 


.38 


.538 


SES X Amount 


.05 


2 


.02 


,11 


.899 


X time 












Residual 


63.04 


288 


.22 






Total 


72.92 


299 


.24 
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Table 3. 

Percent returned by SES and time of payment, collapsed across incentive amount 



SES Prepaid Promised 



High 74 70 

Middle 68 46 

Low 66 30 



Note . Percentages based on n = 50 per cell. 



